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DISTRIBUTED  PROBLEM  SOLVING: 

ADAPTIVE  NETVORKb  WITH  A  COMPUTER  INTERMEDIARY  RESOURCE 


Part  One 

GROUP  PROBLEM  SOLVING  PERFORMANCE  IN  A  SIMULATED 
MILITARY  SITUATION  ASSESSMENT  TASK  UNDER  VARYING 
ENVIRONMENTAL  CONDITIONS 


Abstract 

Networking  of  information  sources  in  relation  to  a  an  organizing  center  is  often  a  requirement  for 
approaching  solutions  to  real  world  problems  that  no  lone  human  can  solve.  Part  of  the  process  may  require 
ongoing  situation  assessment  by  the  problem  solving  participants  and  the  sharing  of  assessment  progress  via 
transfer  of  information  over  physically  restricted  communication  links.  Often,  the  communication  process 
may  occur  under  conditions  where  the  participants  may  be  personally  stressed  in  various  ways.  In  the 
experiment  reported  here,  three,  physical  partition  separated,  Macintosh  computer  workstations  were  placed 
in  mutual  communication  under  various  restrictions  arxf  monitored  by  a  fourth  computer  that  served  as  a  file 
sen/er  and  central  data  collection  resource.  The  problem  situation  was  presented  as  antimated  tanks  moving 
across  a  master  battlefield  terrain  map  with  different,  overlapping,  limited,  portions  of  the  map  shown  on  the 
screen  at  each  workstation.  With  various  restrictions  placed  on  the  communication  channels,  subjects  sent 
and  received  messages  and  ‘jied  to  arrive  at  conclusions  regarding  tank  action  parameters.  Twenty-four 
college  students  participated  as  subjects.  A  10%  and  a  50%  probability  of  detection  of  the  observer  by  the 
"enemy  "  produced  a  threat  situation  and  certain  operational  consequences  if  detection  occurred.  The  objective 
was  to  am've  at  an  esJmate  of  enemy  strength,  destination,  tactica'  objective,  and  observer  location.  The  results 
suggest  that  a  broadly  accessible  communications  system  produces  higher  quality  situation  assessment  results 
but  has  certain  constraints  and  caveats. 


1.0  INTRODUCTION 

In  order  to  generate  a  viable  solution  for  many  real  world  problems,  a  broad  spectrum 
of  information  is  usually  required — a  spectrum  that  exceeds  the  range  and  capabilities  of  a 
lone  human  problem  solver  or  a  single  source  of  information.  When  such  a  situation  exists, 
the  basic  processes  for  problem  solving,  such  as  effective  option  generation  and  situation 
assessment,  may  require  substantial  broadening  of  database  and  problem  solving  re¬ 
sources  Namely,  a  number  of  experts  with  different,  yet  necessary  resources  are  required 
to  solve  the  problem.  Each  of  the  experts  involved  with  the  problem  will  presumably  have 
differing  areas  of  knowledge  or  expertise,  and  this  is  significant  in  that  the  experts  must 
communicate  and  cooperate  with  each  other  in  the  sense  that  not  one  of  them  alone  has  the 
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sufficient  resources  to  solve  the  entire  problem;  mutual  sharing  of  information  is  necessary 
to  allow  the  group  as  a  whole  to  produce  a  solution.  Therefore,  in  a  distributed  problem 
solving  situation  the  knowledge,  experlibe,  and  information  needed  for  dealing  with  the 
problem  will  be  found  distributed  among  several  experts.  Furthermore,  for  the  problem  to 
be  properly  dealt  with,  the  experts  must  work  together  as  a  team.  This  is  the  distributed 
problem  solving  concept,  and  using  humans  and  appropriate  support  equipment  can  be  an 
effective  method  for  accessing  such  a  broadened  database  and  generating  a  viable  solution. 
How  humans  interact  with  each  other  while  using  computers  as  intermediary  resources  for 
distributed  problem  solving  tasks,  however,  remains  open  to  many  research  questions. 

The  concept  of  team  effort  is  one  of  the  most  important  aspects  of  distributed  problem 
solving.  A  team  can  be  defined  as  a  group  of  two  or  more  people  who  are  working  towards 
a  common  goal/objective/mission,  where  each  person  has  been  assigned  specific  roles  or 
functions  to  perform,  and  where  completion  of  the  objective  requires  some  form  of 
dependency  among  tf;'j  group  members  (Dyer,  1984).  Without  a  successful  team  effort,  a 
distributed  problem  solving  task  cannot  be  dealt  with  in  an  efficient  and  effective  manner. 

Many  real  world  problems  fit  into  the  distributed  problem  solving  realm.  In  general, 
situation  assessment,  disaster  management,  medical  diagnosis,  business  management, 
product  research  and  development,  operation  of  complex  systems,  and  C^/C^l  (command, 
control/command,  control,  communication,  intelligence)  environments  all  lend  themselves 
to  the  distributed  problem  solving  process.  In  each  of  these  situations,  a  number  of  human 
experts  ijnction  as  a  team  to  achieve  some  desired  objective,  using  computers  and 
appropriate  support  equipment  to  assist  them  in  performing  their  work. 

With  growth  in  microprocessor  technology,  network  technology,  and  the  general 
complexity  of  many  types  of  problems,  it  has  been  argued  that  problem  solving  tasks  in 
certain  distributed  problem  solving  situations  should  be  entirely  automated.  This  argument 
is  further  bolstered  by  the  vast  amount  of  empirical  psychological  data  showing  both  the 
limitations  of  human  information  processing  capabilities  and  the  effects  of  environmental 
pressures  on  these  capabilities  (e.g.,  Wickens,  1984;  Jacobs,  1984;  National  Research 
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Council,  1982;  Dawes,  1979;  Shroeder&Benbasat,  1975;  Dawes  &  Corrigan,  1974;  Wright, 
1 974;  Folkins,  1 970;  Nomikos  et  al.,  1 968;  Monat  et  al.,  1 972;  Hayes,  1 964).  For  example, 
Hutchins  et  al.  (1984)  studied  human  performance  in  a  military  command  and  control  (C^) 
environment  during  a  simulated  air  defense  operation  and  found  that  in  high-density 
situations  operators  were  highly  productive  but  decidely  suboptimal  in  their  performance. 
That  is,  in  a  situation  where  operators  were  overloaded,  their  performance  was  highly 
productive  as  measured  by  the  number  of  missies  they  launched,  while  on  the  other  hand, 
their  performance  was  significantly  suboptimal  (e.g.  use  of  poorer  decision  making  strate¬ 
gies  as  the  situation  grew  more  complex)  as  measured  by  other  indices. 

Given  the  limitations  in  human  information  processing  capabilities,  many  researchers 
have  suggested  the  development  of  automated  systems  to  contend  with  the  distributed 
problem  solving  process.  Lesser  and  Corkill  (1981 )  propose  a  general  architecture  for  an 
automated  distributed  processing  system  to  deal  with  distributed  interpretation,  distributed 
network  traffic-light  control,  and  distributed  planning.  In  Lesi'er  and  Corkill’s  approach, 
computerized  nodes  cooperatively  problem-solve  by  exchanging  partial  tentative  results  (at 
various  levels  of  abstraction)  within  the  context  of  common  goals.  Wesson  et  al.  (1 981 )  and 
Ben-Bassat  and  Freedy  (1982)  propose  that  situation  assessment  tasks  involved  with 
military  applications  can  and  should  be  fully  automated  by  using  a  distributed  sensor 
network.  These  authors  put  forth  a  general  architecture  for  an  expert  systerr,  and  its 
associated  sensor  network  that  would  perform  problem  solving  tasks  in  a  distributed 
situation  assessment  setting.  In  other  papers,  protocols  for  control,  communication,  and 
cooperation  between  expert  system  sensor  nodes  in  a  distributed  problem  solving  network 
have  been  described  (Smith  &  Davis,  1981 ;  Yang  et  al.,  1985). 

Testbed  systems  and  models  have  been  developed  for  initial  studies  of  automated 
distributed  problem  solving  networks  in  detecting  and  tracking  low  flying  aircraft  (Lacoss  & 
Walton,  1978),  speech  understanding  (Erman  et  al.,  1980),  situation  assessment  (Wesson 
et  al.  1 981 ),  vehicle  monitoring  (Lesser  et  al.,  1 982),  and  supervisory  control  (Govindaraj  et 
al.,  1 985).  Research  such  as  this  is  aiming  for  an  artificial  intelligence  (Al)  system  capable 
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of  distributed  problem  solving. 

In  the  initial  development  of  their  Al  testbeds,  however,  it  is  disturbing  to  note  that  a 
number  of  researchers  have  attempted  to  remove  the  central  focus  of  any  proNtm  solving 
situation — ^the  human  element.  Statler  (1 984),  for  example,  has  observed  in  the  design  and 
development  phases  of  many  complex  human-machine  systems  that  many  engineers  either 
ignore  or  do  not  understand  the  fundamental  requirements  of  human  senscfy,  cognitive,  and 
physical  characteristics  and  limitations,  that  is,  they  do  not  take  into  account  the  true  needs 
and  attributes  of  the  potential  human  user  population.  Not  all  problem  solving  tasks  in  a 
distributed  problem  solving  situation  can  be  automated.  In  fact,  it  is  not  feasible  at  this  time, 
nor  even  in  the  near  future,  to  totally  automate  the  human  out  of  a  complex  problem  solving 
situation  such  as  disaster  management  or  command/control  environments.  Different  levels 
of  human  performance  (sensory,  cognitive  and  physical)  can  be  vastly  improved  with 
automated  assistance,  however,  automation  cannot  satisfactorily  replace  human  capabili¬ 
ties  entirely.  Andriole  and  Haplin  (1986)  have  stressed  the  important  role  of  humans  in  the 
command/control  process  by  stating  that  unless  research  and  development  attention  is 
focused  on  the  human  element  in  the  process,  then  it  will  not  matter  how  sophisticated 
the  automated  systems  are.  This  statement  can  be  generalized  to  many  other  types  of 
systems.  Advantages  and  disadvantages  of  automation  in  both  complex  human-machine 
systems  and  problem  solving  situations  have  received  much  attention  in  human  factors 
engineering  studies  (for  example,  see  Edwarcis,  1977;  Wiener  &  Curry,  1980;  National 
Research  Council,  1982;  Mitchell,  1982;  Boehm-Davis  et  al.,  1983;  Lederer,  1983;  Melvin, 
1 983;  Wiener,  1 983;  Wickens,  1 984;  Helander,  1 985;  Price,  1 985;  Kearsley  &  Seidel,  1 985; 
Weiner,  1985;Parsons,  1985;  Sheridan,  1986;Kantowitz&Sorkin,  1986,Czaja,  1986;Davis 
&  Wacker,  1986). 

While  the  efforts  to  automate  some  asoects  of  a  distributed  problem  solving  task  appear 
to  be  a  promising  direction  for  research  and  development,  many  of  the  requirements  in  these 
problem  situations  will  remain  heavily  dependent  upon  human  skills.  Flexible,  creative 
thinking  and  the  ability  to  reason  inductively  are  important  facets  of  distributed  problem 
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solving,  and  at  this  point  in  time,  there  is  no  computer  capable  of  performing  these  functions. 
Therefore,  humans  will  remain  the  ce"tral  figure  in  the  problem  solving  process.  Rouse  and 
Morris  (1986)  point  out  that  while  automation  can  result  in  the  improvement  of  task 
performance  and  productivity,  in  addition  to  increasing  organizational  control,  automated 
systems  can  also  give  rise  to  many  types  of  unanticipated  risks  and  problems.  Fc .  instance, 
excessive  downtime  for  repairs  and  calibrations  occur  with  automated  systems. 

An  illustrative  example  of  unanticipated  problems  with  Al  systems  was  provided  by 
TRW’s  “adept  work  station  project”  (Technical  Survey,  1 986).  TRW  explored  the  use  of  Al 
in  battlefield  situation  assessment  (SA)  under  AWSP.  In  attempting  to  make  Al  solve 
practical  GA  problems,  the  researchers  at  TRW  were  forced  to  re-examine  some  of  the 
traditional  beliefs  associated  with  the  Al  field.  One  traditional  belief,  that  of  machines 
reason  by  themsekes,  was  'ejected  by  TRW.  The  researchers  found  that  Al  offers  no  more 
reasoning  capabilitiy  than  conventional  computer  algorithms,  and  that  humans  are  required 
when  reasoning  is  required,  particuL-iily  for  unexpected  events.  The  human  skills  of 
improvising  and  using  flexible  procedures  allow  the  human  to  handle  unexpected  events 
while  the  computer  is  quite  incapable  of  doing  so. 

TRW’s  AWSP  also  showed  the  information  processing  problems  that  human  operators 
can  enccunterwhen  using  an  Al  SA  system.  One  of  the  systems  that  was  being  developed 
under  AWS  ^  was  called  the  Battlefield  Exploitation  and  TargOi  Acquisition  (BETA)  system, 
a  computerized  situation  assessor.  It  was  a  prototype  designed  to  collect  a  variety  of 
oattlefield  information,  put  it  together,  and  display  it  to  intelligence  analysts.  Initial 
experiments  with  BETA  showed  that  analysts  could  quickly  become  overwhelmed  with  data, 
and  a  later  experiment  using  army  officers  obtained  similar  results.  In  another  experiment 
with  army  officers,  the  officers  had  complete  control  over  the  amount  of  information  they 
received  from  BETA.  At  first,  the  officers  relied  on  intuition  and  uo  data,  and  lost  the  battle. 
On  the  second  run,  they  received  all  the  data  the  system  could  deliver,  became  overloaded, 
and  lost  the  battle.  In  between  these  two  extremes  lies  the  ideal  amount  of  data,  however, 
this  amount  can  vary  with  individuals. 
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To  overcome  this  human  information  processing  overload,  Al  researchers  attempted  to 
develop  machine  reasoning  and  flexible  software  technology.  This  technology  was 
implemented  into  AWSP,  and  BETA  was  to  be  capable  of  analyzing  intelligence  data  on 
Soviet  second-echelon  forces,  thereby  dec; easing  the  cognitive  demands  placed  on  the 
analysts.  Unfortunately,  BETA  did  not  meet  its  objectives  and  was  limited  to  deciding  when 
Soviet  vehicles  would  leave  the  road.  Furthermore,  the  system  had  a  limited  knowledge 
base  that  was  difficult  to  modify,  and  the  interface  between  soldier  and  machine  was  slow 
and  clumsy. 

With  the  present,  state-of-the-art  Al  technology.  SA  will  remain  a  human-based  task,  as 
with  many  other  distributed  problem  solving  tasks.  Although  many  of  today’s  Al  and  expert 
system  programs  are  impressive  in  a  numberof  ways,  they  are  not  yet  ready  to  take  on  critical 
tasks  alone.  Lehner  ( 1 986)  has  reviewed  the  application  of  Al  techniques  to  computer-based 
support  of  command  and  control  decision  making  and  has  concluded  that  these  military 
expert  systems  operate  only  as  intelligent  interfaces  ratherthan  stand  alone  systems.  In  the 
foreseeable  future,  such  programs  will  be  used  to  assist  human  problem  solvers  in  dealing 
with  difficult  and  complex  tasks.  Projects  such  as  TRW’s  AWSP  are  making  important 
findings  which  are  being  applied  to  experimental  computer  systems  that  can  aid  the  analysts 
in  an  SA  task.  As  of  now,  the  ability  of  the  computer  to  function  as  an  SA  analyst  in  itself  is 
very  limited,  and  thus  SA  will  be  heavily  dependent  on  human  skills  for  the  time  being. 
However,  as  a  diagnostic  tool  and  data  enhancer/processor,  the  computer  has  significant 
values  as  an  SA  aid.  There  is  no  doubt  that  computer  and  electronic  techonology  will  by 
playing  very  major  roles  in  tomorrow’s  battlefield  by  assisting  commanders  and  soldiers  in 
gathering,  categorizing,  analyzing,  and  distributing  important  data. 

From  this  brief  discussion,  it  is  clear  that  the  role  of  the  human  in  a  distributed  problem 
solving  task  is  secured.  Since  the  technological  trend  is  pushing  towards  implementing  more 
automation  and  computer  assistance,  the  need  to  understand  and  enhance  user  perform¬ 
ance,  user  interaction,  and  user  acceptance  ofcomputertechnology  is  imperative.  Further¬ 
more,  having  knowledge  of  human-system  performance  characteristics  in  distributed 


problem  solving  situations  can  significantly  enhance  future  management  of  such  problems. 
The  overall  objective  of  this  study  was  to  identify  and  characterize  some  of  the  human 
problem  solving  behaviors  that  emerge  when  a  computer  intermediary  resource  is  made  an 
integral  part  of  a  distributed  problem  solving  situation. 

There  were  two  specific  purposes  of  this  study.  First,  to  study  human  communication 
in  both  stressed  and  unstressed  situations,  and  second,  to  examine  how  different  group 
communication  protocols  influence  situation  assessment  performance. 

A  simulated  battlefield  situation  assessment  task  was  chosen  as  the  distributed  problem 
solving  task.  This  was  done  for  a  number  of  reasons.  First,  battlefield  situation  assessment 
requires  more  than  one  person  to  carry  out  the  required  tasks.  The  SA  group  is  a  team,  i.e. 
all  members  have  a  common  objective — ^to  assess  the  overall  battlefield  situation.  Second, 
SA  groups  are  often  decentralized,  and  information  processing  and  decisionmaking  are 
distributed  (Cohen  et  al.,  1 985).  Third,  a  battlefield  situation  assessment  environment  can 
impose  some  severe  environmental  constraints  that  can  affect  the  performance  of  the  SA 
group.  Some  real  world  environmental  constraints  such  as  time  limits,  chances  of  enemy 
detection,  and  battlefield  complexity  can  be  implemented  into  the  computer-simulated 
battlefield  scenarios.  Finally,  this  study  was  supported  by  the  U.  S.  Army  Research  Institute 
under  Contract  MDA903-84-C-0355  with  the  UCLA  Man-Machine-Environmental  Engineer¬ 
ing  Laboratory,  and  it  seemed  appropriate  to  employ  a  distributed  situation  assessment  task 
that  was  directly  related  to  army  matters. 

2.0  EXPERIMENT 


2.1  Method 

2.1.1  Overview.  This  distributed  situation  assessment  experiment  was  designed  to 
study  group  problem  solving  behavior  in  a  simulated,  dynamic  battlefield  situation  assess¬ 
ment  task  while  using  the  computer  as  an  intermediary  resource.  Specifically,  this  research 


7 


examined  how  specific  group  communication  protocols  employed  under  different  environ¬ 
mental  conditions  influenced  distributed  situation  assessment  performance. 

The  general  experimental  environment  consisted  of  three,  partition-separated  human 
subject  workstations,  each  containing  a  computer  that  was  mutually  networked  according  to 
specific  experimental  protocols.  A  fourth  computer  workstation  functioned  as  the 
experimenter’s  workstation.  The  experimenter’s  computer  system  provided  a  mutually 
accessible  file  server,  data  management  and  data  collection  resource.  For  these  studies, 
Apple  Macintosh  Plus  computers  were  used,  giving  versatile  capabilities  for  employing 
dynamic  graphics,  i.e.  animated  displays  and  maps,  for  studying  distributed  situation 
assessment  performance.  Subjects  in  a  group  saw  different  parts  of  a  battlefield  and  had 
to  formulate  an  overall  picture  of  what  was  going  on  in  the  entire  battlefield.  Each  subject 
received  only  a  subset  of  the  total  set  of  information  needed  to  assess  the  entire  scenario. 
Solution  to  the  assessment  problem  required  each  subject  to  organize  and  integrate  their 
information  subset  with  additional  information  received  through  communication  with  other 
subjects.  Subjects  interfaced  with  the  problem  situation  and  interacted  with  the  other 
subjects  through  their  respective  computer  systems. 

2.1.2  Subjects.  Twenty-four  undergraduates  (1 6  males  and  8  females)  at  the  University 
of  California,  Los  Angeles  participated  in  this  study  for  course  credit  on  a  voluntary  sign-up 
basis.  Fo^each  experimental  run,  subjects  were  arranged  into  groups  of  three,  thus  making 
a  total  of  eight  groups.  The  subjects’  ages  ranged  from  1 8  to  25  years.  Most  had  little  or  no 
experience  with  Macintosh  computers. 

2.1.3  Materials  and  apparatus.  A  computer-controlled  distributed  situation  assessment 
environment  was  developed  using  one  Apple  Macintosh  Plus  computer  coupled  to  an  80 
megabyte  hard  drive  and  the  Pascal  programming  language  to  drive  three  other  Macintosh 
Plus  computers  in  separated,  partitioned  workstations.  The  main  Macintosh  was  the 
experimenter’s  computer  and  was  designated  MAC  0.  The  other  three  Macintoshes  were 
designated  MAC  1,  MAC  2,  and  MAC  3.  MACs  1-3  were  assigned  to  subjects  A-C, 
repsectively.  Figure  1  displays  the  Macinto»:h  workstation  layout.  All  four  computers  were 
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4  Macintosh  computers,  Imagewriter  II  printer, 
and  Apple  Talk  network 

Figure  1.  Macintosh  Workstation  Layout 


Figure  2.  General  Layout  of  Subjects's  Macintosh  Screen 
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connected  with  the  Appletalk  network. 

Each  Macintosh  computer  screen  contained  information  on  time  allotted  for  situation  as¬ 
sessment,  probability  of  detection  by  the  “enemy”  (MAC  0),  messages  being  sent  and 
received,  and  an  animated  battlefield  display.  Figure  2  illustrates  the  screen  that  subjects 
see  on  their  Macintosh  computers. 

2.1.4  Design.  A  between-subjects  design  was  used.  There  were  three  between- 
subjects  independent  variables:  communication  protocols,  probability  of  detection  of  a 
group’s  message  transmission  by  MAC  0,  and  scenario  complexity. 

The  first  experimental  variable  was  communication  protocol.  The  subject  population 
was  randomly  divided  into  two  different  communication  protocol  modes:  Broadcast  Default 
Communication  (BRODCOM)  and  Selective  Defauit  Communication  (SELCOM). 
Common  to  both  communication  protocol  modes  is  the  general  method  in  which  subjects 
interacted  with  each  other.  In  order  to  communicate  with  one  another,  subjects  had  to  use 
the  keyboard  to  type  in  the  message  and  the  mouse  pointing  device  to  direct  the  message 
to  the  other  subjects.  Forthe  BRODCOM  group,  the  communication  protocol  was  as  follows. 
First,  when  a  subject  sent  a  message,  it  went  to  the  other  two  subjects  simultaneously:  that 
is,  anytime  a  subject  sent  a  message,  the  other  two  subjects  in  the  group  automatically 
received  it.  For  a  subject  to  communicate  with  his  or  her  group  in  the  SELCOM  mode,  he 
or  she  had  to  select  which  of  the  other  two  subjects  to  transmit  a  message  to.  A  subject  in 
a  SELCOM  group  only  received  a  message  that  was  sent  specifically  to  him  or  her  by  another 
subject  at  a  particular  time.  In  both  BRODCOM  and  SELCOM  groups,  subjects  could  chose 
to  remain  autonomous,  i.e.  to  remain  silent  and  not  communicate  with  anyone  in  the  group 
if  the  situation  demanded  it. 

The  experimental  groups  were  randomly  assigned  to  two  different  levels  of  detection 
probability.  In  this  age  of  high  technology  warfare,  electronic  warfare  (EW),  electronic 
countermeasures  (ECM),  and  electronic  counter  countermeasures  (ECCM)  play  a  very 
significant  role  in  the  battlefield.  To  add  a  simple  case  of  EW  to  this  study,  and  also  to  induce 
some  sort  of  environmental  constraint,  probablity  of  detection  was  implemented  into  the 
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computer  system. 

For  a  single  message  transmission,  there  were  two  levels  of  detection:  1 0%  and  50%. 
What  these  probabilities  meant  was  that  each  time  a  subject  transmitted  a  message,  there 
was  X  probability  that  the  “enemy”  (the  experimenter’s  computer)  would  detect  the  transmis¬ 
sion.  The  probability  of  detection  was  straight  forward.  Depending  upon  which  detection 
level  was  used,  each  time  a  message  was  transmitted  there  was  eithera  1 0%  or  50%  chance 
of  enemy  detection.  For  example,  if  the  10%  detection  probability  level  was  used,  each  time 
a  message  was  transmitted  there  was  a  10%  chance  that  the  “enemy”  detected  this 
transmission.  The  “enemy”  (MAC  0)  made  these  detections  through  a  random  number 
generation  process.  The  consequences  of  a  detection  are  explained  in  the  procedure 
section.  Subjects  in  both  BRODCOM  and  SELCOM  groups  were  informed  of  these 
probability  differences.  With  respect  to  this  independent  variable,  the  three  subjects  in  a 
situation  assessment  group  had  to  weigh  tradeoffs  between  the  need  for  information  and  the 
probability  of  being  detected  by  the  enemy. 

The  third  independent  variable  was  battlefield  scenario  complexity.  There  were  three 
levels  of  complexity — low,  medium,  and  high.  Scenario  complexity  was  defined  as  the 
number  of  elements  in  the  battlefield.  Elements  consisted  of  enemy  (Red  Force)  tanks  and 
type  of  battlefield  terrain.  The  low-level  complexity  scenario  was  used  as  a  demonstration 
and  practice  scenario  for  the  subjects,  while  the  medium-  and  high-level  complexity 
scenarios  were  used  for  the  actual  experimental  runs.  Figures  3,  4,  and  5  show  the  maps 
of  the  battlefields  used  for  each  scenario.  The  three  types  of  scenarios  will  be  described  in 
a  later  section. 

Three  dependent  variables  were  used  to  measure  the  effects  of  commit r.ication 
protocols  on  situation  assessment  performance:  1)  accuracy  of  situation  assessment,  2) 
subjective  workload  ratings  of  the  situation  assessment  task,  and  3)  the  degree  of  agreement 
between  group  members  on  their  assessments  of  the  battlefield  situation. 

Situation  assessment  performance  was  the  measure  of  the  accuracy  of  a  particular 
group’s  overall  assessment  of  the  situation.  Subjects’  assessments  were  to  contain  the 
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Figure  3.  Battlefield  for  the  Low-Level  Complexity  Scenerio 
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Figure  4.  Battlefield  for  the  Mid-Level  Complexity  Scenario 
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Figure  5.  Battlefield  for  the  High-Level  Complexity  Scenario 
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following  items: 

•  estimate  of  enemy  strength  (i.e.  number  of  tanks) 

•  estimate  of  enemy’s  destination  (e.g.,  town,  roadfork,  bridge) 

•  estimate  of  enemy’s  objective  (e.g.,  surround  town,  attack  installation) 

•  estimate  of  the  postions  of  the  other  subjects 

Accuracy  reflected  how  precise  the  subjects  were  in  their  assessment  of  the  above 
categories  (enemy  strength,  destination,  objective,  number  of  detections,  and  position  of 
other  subjects)  as  compared  to  the  actual  information  contained  in  each  of  the  categories. 

These  estimates  are  important  elements  of  situation  assessment.  Military  planners 
must  know  as  much  as  possible  about  the  enemy’s  disposition  and  activities.  From  this 
information,  planners  can  derive  estimates  on  enemy  capabilities  and  intentions  and 
incorporate  these  into  their  plans.  Situation  assessment  (or  situation  analysis)  is  an 
extremely  important  element  of  battlefield  planning  because  it  is  in  this  assessment  process 
that  the  capabilities  of  the  enemy  must  be  identified  and  evaluated,  and  intentions  inferred 
so  that  appropriate  courses  of  action  can  be  taken  by  friendly  forces  (Loeberg  et  al.,  1 986). 
Capabilities  are  the  courses  of  action  we  believe  the  enemy  can  conduct,  and  intentions  are 
the  most  likely  courses  of  action  we  believe  the  enemy  will  carry  out.  The  four  estimates 
above  were  believed  to  be  capable  of  providing  the  necessary  information  to  the  subjects  on 
enemy  capabilities  and  intentions  all  within  the  context  of  this  study. 

The  second  dependent  measure  was  subjective  workload  ratings.  Subjects  rated  the 
relative  workload  of  each  situation  assessment  session  based  on  nine  items  adapted  from 
Hart  et  al.  (1984).  The  nine  items  were:  overall  workload,  task  difficulty,  time  pressure, 
performance,  mental  and  sensory  effort,  physical  effort,  frustration  level,  stress  level,  and 
fatigue. 

The  third  measure  examined  the  level  of  agreement  a  subject  had  for  the  assessments 
of  the  other  two  subjects  in  the  group. 

Battlefield  scenarios.  As  mentioned  above,  there  were  three  different  battlefield 
scenarios  used  in  this  study:  low-level  complexity,  mid-level  complexity,  and  high-level 
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Figure  6.  Key  Battlefield  Landmarks  for  the  Low-Level  Complexity  Scenerio 
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complexity.  The  low-level  complexity  was  used  for  a  demonstration  and  practice  seesion  to 
get  subjects  familiarized  with  the  operational  requirements  and  characteristics  of  their 
computer  systems.  The  medium-  and  high-level  complexity  scenarios  were  used  for  the 
actual  experimental  runs. 

In  the  low-level  complexity  (LLC)  scenario  (see  Figure  6),  a  total  of  five  Red  Force 
(enemy)  tanks  were  employed.  A  column  of  3  tanks  entered  the  main  battle  area  (MBA)  from 
the  northeast  and  proceeded  westward  on  the  main  east-west  road  that  took  them  across 
the  river.  Later,  a  column  of  2  tanks  emerged  onto  the  MBA  from  the  northwest  and  traveled 
south  on  the  road  located  directly  west  of  the  river.  These  two  tanks  rolled  down  this  road 
to  the  main  road  intersection  and  stopped  there  to  wait  for  the  other  three  tanks.  The 
objective  for  these  two  Red  Force  tank  columns  were  to  meet  up  with  each  other  at  the  major 
crossroads  intersection  on  the  western  side  of  the  MBA  and  then  proceed  westward. 

For  the  mid-level  complexity  (MLC)  scenario  (see  Figure  7),  1 0  Red  Force  tanks  were 
used.  The  10  tanks  were  divided  into  three  columns;  Red  Column  One  contained  3  tanks 
,  Red  Column  Two  had  3  tanks,  and  Red  Column  Three  had  4  tanks.  The  scenario  unfolded 
as  follows.  Red  Column  One  (Red  1  -3)  came  out  of  the  southwest  MBA  corner  and  moved 
up  the  road  until  it  passed  clear  of  the  hill  which  nestles  the  eastern  side  of  the  road.  At  this 
point  the  tanks  left  the  road  and  rumbled  into  the  countryside,  heading  east  toward  the  town. 
About  halfway  between  the  road  and  the  town,  the  tanks  stopped  and  positioned  themselves 
so  that  each  had  its  gun  aimed  toward  the  town. 

When  this  was  done,  the  other  two  tank  columns  appeared.  The  tanks  of  Red  Column 
Two  (Red  4-7)  traveled  up  from  the  south  on  the  road  located  in  between  the  two  rivers. 
Shortly  afterthe  appearance  of  Red  Column  Two  tanks  in  the  MBA,  the  tanks  of  Red  Column 
Three  (Red  8-9)  rumbled  into  the  MBA  and  began  making  their  way  down  from  the  north 
along  the  western  bank  of  the  river. 

Once  Red  Column  Two  made  its  way  to  the  semi-clearing  on  the  western  side  of  the 
road,  the  tanks  turned  left,  eased  off  the  road,  and  headed  toward  the  eastern  bank  of  the 
western  river  fork.  Upon  reaching  the  bank,  they  stopped  and  aimed  their  guns  at  the  town. 


17 


Red  1  -  3  enter  the 
battlefield  here 


Red  7-10  enter 
the  battlefield  here 


Figure  7.  Key  Battlefield  Landmarks  for  the  Mid-Level  Complexity  Scenario 
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The  rvD  Red  F/A 
adrciaft  fly  into 
the  MBA  here 


Red  1-6  enter  the  MBA 
here  and  cross  the  river 
bridge 


Figure  8.  Battlefield  for  the  High-Level  Complexity  Scenario 
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During  this  time,  Red  Column  Three  progressed  downward  along  the  western  bank  of  the 
western  river  fork  toward  the  northern  end  of  the  town. 

The  objective  of  the  enemy  was  to  surround  the  northern,  eastern,  and  western  flanks 
of  the  town  and  prepare  for  an  offensive  thrust.  The  scenario  ended  with  three  tanks  on  the 
western  flank  of  town,  four  tanks  on  its  eastern  flank,  and  three  tanks  making  their  way  down 
to  position  themselves  on  the  northern  flank. 

In  the  high-level  complexity  (HLC)  scenario  (see  Figure  8),  Red  Forces  had  three 
objectives.  First,  to  cross  the  river  and  secure  the  bt  idge.  Second,  to  have  two  tank  columns 
rendezvous.  Third,  to  surround  and  attack  the  Blue  Force  Command/Control  installation. 

Fifteen  Red  Force  tanks  and  two  Red  Force  fighter/attack  aircraft  were  employed  in  this 
scenario.  The  scenario  basically  unfolded  as  follows.  A  column  of  six  Red  Force  tanks  (Red 
Column  One  with  tanks  1  -6)  entered  the  MBA  from  the  south-central  portion  of  the  field  and 
proceeded  north  across  the  bridge.  The  first  two  tanks  across  the  bridge  positione 
themselves  in  the  trees  with  their  guns  pointing  up  at  the  Blue  C^  installation.  The  third  tank 
secured  the  bridge  crossing  by  positioning  itself  at  the  entrance,  and  the  fourth,  fifth,  and  sixth 
tanks  traveled  northeast  up  the  road  to  rendezvous  with  another  tank  column. 

The  three  tanks  traveling  northeast  rendezvoused  with  five  other  Red  Force  tanks  (Red 
Column  Two  with  tanks  7-11)  at  the  road  fork  located  in  the  eastern  portion  of  the  MBA.  When 
the  rendezvous  was  made,  the  eight  tanks  followed  the  road  that  took  them  around  the  lake 
toward  the  Blue  Force  hill.  Prior  to  completely  rounding  the  lake,  the  eight  tanks  stopped  to 
keep  out  of  range  of  Blue  Force  guns. 

Shortly  thereafter,  four  Red  Force  tanks  (Red  Column  Three  with  tanks  12-15)  stormed 
into  the  MBA  from  the  west.  These  fourtanks  converged  on  the  backside  of  the  hill  and  slowly 
climbed  it.  As  the  tanks  began  climbing  the  hill,  the  eight  tanks  parked  along  the  road  lining 
the  lake  started  moving.  When  this  all  occurred,  two  Red  Force  combat  aircraft  streaked  out 
of  the  east  and  flew  on  a  westerly  heading  to  perform  an  air-to-ground  attack  on  the  Blue 
Force  hill. 

The  scenario  ended  with  all  Red  Force  elements  beginning  their  attack  maneuvers. 
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2.1.5  Procedure.  Forth©  situation  assessment  session,  each  of  the  three  subjects  were 
placed  at  their  own  Macintosh  workstation.  The  subjects’  task  was  to  observe  and  assess 
the  developing  situation,  and  then  to  arrive  as  best  as  they  could  at  an  interpretation  of  what 
was  going  on  in  the  entire  battlefield  (not  just  what  was  going  on  in  their  own  field  of  view). 
To  achieve  this  goal,  the  subjects  had  to  communicate  with  each  other  through  their  com¬ 
puter  keyboards:  they  had  to  send  each  other  information  concerning  battlefield  develop¬ 
ments  during  the  situation  assessment  period. 

Subjects  were  run  in  groups  of  three,  and  each  group  was  randomly  assigned  to  the 
experimental  conditions.  Groups  of  three  subjects  were  run  through  a  10  minute  situation 
assessment  assignment  on  either  the  medium-level  or  high-level  complexity  scenario.  For 
every  scenario  run,  each  subject  saw  a  subset  of  the  entire  battlefield  on  their  Macintosh 
screen  (Figures  9-11),  and  their  combined  fields  of  view  did  not  constitute  the  complete 
picture.  In  addition,  the  subjects  did  not  know  who  saw  what.  The  three  different 
perspectives  partially  overlapped,  and  the  orientation  and  amount  of  overlap  remained 
constant  throughout  all  three  scenarios. 

Before  beginning  the  actual  situation  assessment  session,  subjects  were  given  a 
tiiorough  briefing  on  the  objectives,  procedures,  requirements,  and  features  of  the  battlefield 
situation  assessment  task.  Upon  completion  of  the  briefing  session,  the  subjects  were 
seated  at  their  Macintosh  workstations  and  were  run  through  the  low-level  complexity 
scenario.  During  this  time,  subjects  familiarized  themselves  with  the  computer  and  the 
different  features  of  the  assessment  task.  Subjects  could  ask  questions  during  this 
demonstration/practice  session.  This  practice  session  generally  lasted  40  minutes.  When 
this  session  ended,  the  subjects  were  instructed  to  put  on  headphones  in  which  low  level 
white  noise  was  piped  through.  This  was  done  to  mask  out  as  much  extraneous  sounds  as 
possible  (e.g.,  keyboard  sounds,  exclaimations  made  by  subjects). 

After  the  demonstration/practice  session,  the  actual  experimental  session  was  started. 
During  an  assessment  period,  if  the  “enemy”  (MAC  0)  detected  a  message  transmission, 
MAC  0  sent  a  message  to  all  three  subjects.  MAC  O’s  message  read:  “Warning!  The  enemy 
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Subject  B's  F.O.Y. 


Figure  9.  Subject's  Field  of  View  for  the  Low-Level  Complexity  Scenario 
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Subject  B  *3  F.O.V. 


Figure  10.  Subject's  Field  of  View  for  the  Mid-Level  Complexity  Scenerio 
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Subject  B '3  F.O.V. 
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Subject  A'3  F.O  Y. 
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Subject  C's  F.O.  V. 


Figure  11.  Subject's  Field  of  View  for  the  High-Level  Complexity  Scenerio 
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has  p  Dssibly  detected  this  message  transmission.”  This  message  appeared  in  the  subjects’ 
main  viewing  screens  for  five  seconds  and  then  disappeared.  MAC  0  kept  count  on  how 
many  times  a  detection  was  made.  If  the  detection  count  for  the  entire  group  exceeded  a 
total  of  5  during  a  scenario,  the  initial  consequence  was  the  the  blanking  of  the  subjects’  main 
viewing  screens  for  1 5  secnnds.  MAC  0  then  immediately  transmitted  the  following  message 
to  all  three  subjects:  “Your  group’s  message  transmissions  have  been  detected  by  the 
enemy.  The  enemy  has  initiated  strong  electronic  countermeasures.  The  enemy  will  try  to 
blank  out  your  screen  for  several  seconds,  but  you  must  complete  your  mission  despite 
encountering  strong  ECM.”  After  1 5  seconds,  the  message  disappeared  and  the  subjects 
were  returned  to  their  normal  screens  to  continue  on  with  the  situation  assessment  task.  If 
a  message  transmission  was  detected  again,  the  main  screens  went  blank  again  for  10 
seconds,  and  the  message  this  time  was;  “Your  group  has  run  into  some  heavy  electronic 
countermeasures.  Your  screen  will  be  blank  for  several  seconds.”  Hereafter,  this  second 
message  appeared  with  the  blank  screen  each  time  a  message  transmission  was  detected 
by  MAC  0.  The  consequence  for  message  detection,  therefore,  was  blanking  of  the  subjects’ 
main  viewing  screens  and  loss  of  valuable  time  for  situation  assessment. 

When  a  scenario  was  completed,  the  subjects  were  instructed  by  MAC  0  to  do  three 
things.  First,  the  subjects  were  asked  to  individually  assess  the  entire  battlefield  situation 
based  on  the  information  exchanged  during  the  scenario  viewing;  “Based  on  the  information 
you  have,  please  state  what  you  believe  is  going  on  in  the  battlefield.”  The  subjects  were 
required  to  type  in  their  assessment  using  their  Macintosh  keyboards,  and  MAC  0  recorded 
each  subject’s  interpretation. 

For  the  second  task,  a  subjective  workload  rating  form  (adapted  from  Hart  et  al.,  1 984) 
appeared  on  each  MAC  screen.  Using  the  mouse,  subjects  rated  the  situation  assessment 
task  on  nine  items.  After  subjects  completed  the  workload  rating  scales,  they  were  routed 
to  the  third  task  which  asked  the  subjects  to  rate  each  other’s  battlefield  situation  assess- 
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ment:  “Please  rate  how  strongly  you  agree  or  disagree  with  each  of  your  group  member’s 
assessment  of  the  situation.”  The  subjects  were  referred  to  a  “blackboard”  screen.  On  this 
“blackboard”  appeared  the  above  instruction,  the  subjects’  individual  situation  assessment, 
and  associated  rating  scales.  For  a  given  subject,  his  or  her  own  Macintosh  screen  showed 
his  or  her  own  situation  assessment  first  with  the  other  two  just  below.  There  was  no  rating 
scale  for  the  subject’s  own  assessment,  however,  under  each  of  the  other  two  subjects’ 
assessment,  there  was  the  question  “How  strongly  do  you  agree  or  disagree  with  this 
situation  assessment?"  and  just  below  it  the  following  six  point  rating  scale: 

0  1  2  3  4  5 

Strongly  Somewhat  Weakly  Weakly  Somewhat  Strongly 

Disagree  Disagree  Disagree  Agree  Agree  Agree 

Using  the  mouse  again  (simply  through  pointing  and  clicking),  the  subject  rated  his  or  her 
level  of  agreement  with  the  other  two  subjects’  assessment  and  gave  a  brief  reason  to 
support  the  particular  rating.  A  major  reason  for  this  agreement  rating  was  to  compare  the 
degree  of  agreement  or  disagreement  among  group  members  between  BRODCOM  and 
SELCOM  groups.  When  all  of  the  subjects  finished  with  this  task,  the  experimental  session 
was  completed. 


3.0  RESULTS 

Initial  examination  of  the  data  reveals  several  interesting  trends  concerning  the 
influences  of  human-computer  communication  protocols  on  group  situation  assessment. 
When  comparing  the  total  number  of  messages  sent  during  a  situation  assessment  scenario, 
there  was  a  trend  for  Selective  Communication  groups  to  transmit  more  messages  than 
Broadcast  Communication  groups.  Figure  12  illustrates  this.  This  difference  was  even  more 
pronounced  in  the  higher  complexity  scenario.  In  addition,  both  BRODCOM  and  SELCOM 
groups  that  were  assigned  to  the  50%  detection  level  showed  the  tendency  to  have  a  lower 
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Broadcast  Communication  Mode  =  Selective  Communication  Mode 


s6u!;By  0OUBLUJO|j0d  puB  pBO|>iJOM 


29 


Broadcast  Communication  Mode  =  Selective  Communication  Mode 
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Broadcast  Communication  Mode  =  Selective  Communication  Mode 
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total  number  of  message  trar.smissions  than  BRODCOM  and  SELCOM  groups  assigned  to 
the  10%  detection  level.  It  was  not  known  whether  this  was  due  to  the  greater  number  of 
screen  blackouts  by  Mac  0  (tctal  of  27  blackouts  for  50%  groups  versus  total  of  9  blackouts 
for  10%  groups)  or  to  group  perception  that  the  blackouts  will  affect  their  overall  situation 
assessment  performance  score. 

Interesting  trends  were  found  in  the  workload  and  performance  ratings  (see  Figures  1 3 
- 1 6).  Overall,  the  Broadcast  Communication  groups  gave  higher  ratings  for  their  perform¬ 
ance  than  did  the  Selective  Communication  groups.  From  initial  subjective  analyses  of  the 
subjects’  assessments,  BRODCOM  subjects  appeared  to  have  obtained  a  more  complete 
and  accurate  picture  of  the  entire  situation  than  did  SELCOM  subjects.  Communication 
protocols  seemed  to  have  some  influence  on  subjects’  perception  of  the  situation  assess¬ 
ment  task.  Overall,  BRODCOM  subjects  on  the  average  gave  lower  ratings  for  task  difficulty, 
mental  and  sensory  effort,  frustration  level,  and  stress  level. 

The  degree  of  agreement  between  group  members  appeared  to  be  influenced  by 
detection  probability.  In  both  the  mid-level  and  high-level  complexity  scenarios,  agreement 
levels  were  higher  for  subjects  in  the  1 0%  condition  than  in  the  5C%  condition.  For  the  MLC 
scenario,  BRODCOM  subjects  in  the  10%  condition  gave  an  average  agreement  rating  of 
4.5,  on  a  scale  of  0  to  5  (see  procedure  section).  The  value  4.5  roughly  translates  into  a  strong 
agreement  rating.  BRODCOM  subjects  in  the  50%  detection  level  condition  had  an  average 
agreement  level  of  3.0.  This  translates  into  a  weak  agreement  among  subjects.  For 
SELCOM  subjects  in  the  MLC  scenario,  subjects  in  the  1 0%  detection  condition  responded 
with  a  rating  of  3.8,  meaning  an  average  level  of  agreement.  SELCOM  subjects  in  the  50% 
detection  condition  averaged  a  3.0  agreement  level — weak  agreement.  In  the  H  LC  scenario, 
BRODCOM  10%  subjects  gave  an  average  rating  of  3.0  as  compared  to  2.8  given  by 
BRODCOM  50%  subjects.  SELCOM  1 0%  subjects  responded  with  an  average  of  4.0  while 
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SELCOM  subjects  came  with  an  average  agreement  rating  of  3.6. 


4.0  DISCUSSION 

This  was  an  entry  study  designed  to  find  out  what  influences  different  human-computer 
communication  protocols  had  on  group  situation  assessment  performance.  Several 
r  jteworthy  trends  were  found  concerning  message  transmissions,  perceived  performance, 
and  level  of  agreement  between  grcup  members.  Data  on  individual  group  members 
situation  assessments  have  shown  a  tre-  for  BRODCOM  assessments  to  be  more 
complete  and  accu>ate. 

The  basic  experimental  design  can  be  modified  in  order  to  examine  this  issue  of  human- 
computer  influences  on  situation  assessment  in  greater  depth.  More  experimental  variables 
and  better  data  collection  can  be  implemented  into  the  system.  One  specific  item  that  may¬ 
be  changed  is  the  requirements  for  situation  assessment.  More  strict  requirements  may  be 
designed  for  subjects  to  follow  when  they  are  typing  in  their  assessment.  This  may  mako 
the  whole  process  more  standardized  so  that  statistical  analysis  of  situation  assessment 
content  can  be  conducted. 

One  problem  that  was  encountered  during  data  collection  was  the  reliability  of  subjects 
to  show  up  for  the  experimental  session.  For  each  session,  three  subjects  were  needed.  In 
a  number  of  instances,  only  one  or  two  subject«:  arrived  for  the  experiment.  When  this 
occurred,  a  scramble  for  additional  subjects  ensued;  if  none  could  be  found,  the  session  was 
written  off.  Methods  of  ensuring  more  reliable  subject  turnout  are  being  looked  into. 

Some  significant  trends  have  been  observed  in  this  initial  study  of  human-computer 
situation  assessment.  Further  study  in  this  area  has  the  potential  to  provide  data  that  can 
help  establish  guidelines  to  assist  the  Army  in  finding  optimal  communications  protocols  and 
organizational  structures  forgathering  and  analyzing  battlefield  information,  in  addition  to 
making  the  most  effective  use  of  interactive,  mutually  assistive,  human-computer  resource 
combinations. 
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Final  Report-Part  Two 


GROUP  ACQUISITION  OF  DYNAMIC  CONTROL  SKILLS  IN  A 
FLUID  LEVEL  ADJUSTMENT  PROBLEM 


Abstract 


Experiments  were  conducted  in  a  distributed  problem-solving  context  that  focused  on 
supervisory  process  control  and  the  study  of  group  problem-solving  and  decision-making  in  a 
computer  generated  dynamic  scenario  with  limited  observability  and  controllability.  The  experiments 
synthesized  observations  from  two  sources,  viz.,  single  operator  control  simulations  of  human 
process  control  problem-solving  and  acquisition  of  process  control  skills.  Experimental  questions 
concerned  the  manner  in  which  variation  in  partitioned  observables  and  controllables,  stmcture  and 
number  of  communications  pathways  and  workload  variables  affected  individual  performance. 

The  primary  focus  and  goal  of  the  study  was  toward  enhancing  current  understanding  of 
human  behavior  in  slow  response  systems.  It  was  also  directed  toward  aiding  with  concepts  for  the 
design  and  analysis  of  distributed  problem-solving  and  decision-making  networks. 

The  experiments  placed  three  subjects  in  command  of  a  process  control  problem.  Each 
subject  interacted  with  the  group  and  the  situation  through  his/her  respective  computer  workstation. 
The  problem  situation  was  purposely  created  to  minimize,  but  not  trivialize,  the  learning  needed  by 
subjects  as  they  became  familiar  with  their  assigned  roles  within  the  network.  Further,  their  chan¬ 
nels  of  communication  and  permitted  actions  were  restricted  and  structured  to  provide  data  for  a 
statistical  evaluation  of  the  performance  of  each  group,  per  trial,  with  variations  in  the  scenario 
stmcture.  The  variations  included  the  communication  hierarchy,  the  number  of  control  input  paths, 
and  the  observability  and  controllability  partitioning.  Dependent  variables  included  a  time  integrated, 
squared,  regulation  error,  message  arid  control  input  frequencies  and  serial  connectivity  (action 
transition  matrices),  and  post-trial  subject  surveys. 

This  research  confirmed  and  extended  the  body  of  understanding  of  humans  in  distributed 
problem-solving  situations  by  finding  that:  1)  chain  groups  reached  their  respective  “steady-state” 
final  time  performance  values  in  fewer  trials  and  had  denser  communication  transition  matrices  than 
circle  groups;  2)  chain  groups  tended  to  focus  on  short  term  control  strategies;  3)  a  very  small 
increase  in  the  number  of  control  input  paths  decreased  group  performance  by  overloading  the 
operators;  and  4)  individuals  at  “hub”  communication  stations  showed  the  most  positive  attitudes  and 
tended  to  evolve  leadership  roles. 


1.0  INTRODUCTION 


The  motivation  for  this  investigation  was  to  attempt  to  develop  a  better 
understanding  of  the  ways  and  the  degree  that  the  acquisition  of  distributed  dy- 
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namic  control  skills  of  a  group  is  affected  by:  a)  the  variation  in  partitioned  observ¬ 
ables  and  controllables;  b)  the  communication  network  of  the  group:  c)  the  number 
of  control  input  paths.  The  experiments  synthesized  the  work  of  Morris,  Rouse  and 
Path,  1985;  Morris  and  Rouse,  1985;  Knaeuper  and  Rouse,  1985;  Mann  and  Ham¬ 
mer,  1986,  in  human  problem-solving  in  process  control  using  a  single  operator 
simulation  called  “PLANT  with  the  work  of  Moray,  Lootsteen,  and  Pajeck,  iS86,  on 
acquisition  of  process  control  skills.  The  latter  work  used  a  simulation  extension  of 
the  method  of  Crossman  and  Cook,  1974.  The  results  were  based  on  a  single 
operator  situation  and  were  described  with  action  transition  graphs.  In  the  present 
report  we  have  attempted  to  extend  the  body  of  understanding  of  humans  in  distrib¬ 
uted  problem-solving  situations,  particularly  the  work  reported  by  Bavelas,1951,  and 
by  Parsons,  and  others  at  RAND,  1972,  by  partitioning  the  observables  and  con¬ 
trollable  states  and  inputs  of  the  process  among  a  networked  group  of  operators. 


Figure  1.  Flow  Diagram  of  Closely  Related  Research 
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Insights  were  provides  by  the  works  of  Wesson,  et  al.,  1 981 ,  and  Lesser  and  Cork- 
ill,  et  al,  1982, 1983.  Fundamentals  in  process  control  were  drawn  from  Leigh, 

1 985.  Figure  1  illustrates  the  relationships  of  the  research  with  a  digraph. 


2.0  EXPERIMENT 


The  subjects  were  placed  in  isolation  cubicals  with  headphones  playing 
white  noise.  From  these  workstations,  all  communication  within  the  group  and  all 
observations  and  control  inputs  were  made  to  the  participants  through  Macintosh 
computers  as  shown  in  Figure  2.  The  simulation  had  either  chain  or  circle  commu- 


Macintosh  Workstation  Setup 


MAC  3 
Subject  C 


MAC  2 
Subject  B 
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i 

Experimenter's  Workstation  -  MAC  0 


MAC  1  -  Subject  A 

Figure  2.  Macintosh  Workstation  Setup 
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a.  Circle  -  Horizontally  Partitioned  Observables  and  Controllables 


b.  Chain  -  Horizontally  Partitioned  Observables  and  Controllables 


c.  Chain  -  Vertically  Partitioned  Observables  and  Controllables 


LEGEND 

Fi  =  OBSERVE  PIPE  FLOWS  EXITING  TANKS  OF  ROW  i 
Ti  =  OBSERVE  TEMPERATURE  OF  TANKS  OF  ROW  i 
V|  =  OBSERVE  VOLUME  OF  TANKS  OF  ROW  i 
Si  =  OBSERVE  PIPE  FLOW  EXITING  SOURCES  OF  ROWi 
E  =  OBSERVE  ENERGY  EOORO  RATE  OF  SINKS  OF  ALL  ROWS 
Ci  =  CONTROL  PIPE  FLOWS  EXITING  SOURCES  OF  ROWi 


Figure  3.  Communication,  Observation  and  Control  Structures: 
a)  Structure  i,  b)  Structure  II  and  c)  Structure  III 
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Figure  4.  Overall  Layout 


nication  structures.  Figure  3  shows  how  the  observables  and/or  controllables  were 
partitioned  either  horizontally  or  vertically.  A  “god’s  eye-view”  of  the  simulation  is 
shown  in  Figure  4.  Heated/cooled  “water”  traveled  from  sources  on  the  far  left 
through  pipes  and  intermediate  tanks  to  sinks  on  the  far  right.  Above  each  source 
icon  was  the  source  identification  number  and  the  temperature  of  the  exiting  water. 
Above  each  tank  icon  was  the  tank  identification  number  and  the  temperature  of  the 
exiting  water.  Above  each  sink  was  the  sink  identification  number  and  the  error  in 
the  exiting  energy  flow  rate.  Subjects  were  instructed  to  drive  the  error  values  to 
zero  as  soon  a  possible  and  keep  them  there  throughout  each  of  the  twelve  minute 
trials. 

The  information  was  partitioned  such  that  subjects  only  observed  and/or 
controlled  a  portion  of  the  dynamic  scenario.  Figure  5  shows  an  example  of  what 
subject  station  A  might  have  seen  one  hundred  and  seven  seconds  into  the  simula- 
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Figure  5.  Subject  station  A,  Low  Pipe  Count 


tion.  It  is  clear  that  communication  on  the  network  was  vital  to  the  sharing  of  infor¬ 
mation  in  order  to  successfully  drive  and  maintain  the  error  signals  at  a  value  of 
zero. 

The  dynamics  the  simulation  were  based  on  first  order  flow  models  for  an 
idealized  fluid  with  as  many  as  three  input  pumps  and  three  output  pumps.  The 
conceptual  system  in  shown  in  Figure  6.  Pipe  flows  could  be  controlled  by  the 
subjects  using  the  mouse  and  pointing  and  clicking.  The  block  diagrams  for  the 
energy  and  volume  states  are  shown  in  Figures  7  and  8  respectively.  For  simplicity, 
where  E  =  k  V  T,  the  value  of  k  was  set  to  unity.  Also,  to  conserve  mass,  the  pipe 
flow  into  a  tank  was  shut  off  when  the  volume  maximum  was  achieved.  The  pipe 
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Figure  8.  Volume  State  Equation  Block  Diagram 


To  establish  a  performance  index,  the  following  definitions  are  made: 
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flow  out  of  a  tank  was  shut  off  when  the  volume  minimum  of  the  tank  was  achieved. 
Thus,  the  simulation  had  “fast”  linear  dynamics  and  both  “fast”  and  “slow"  nonlinear 
dynamics. 

After  each  trial,  a  survey  was  taken  by  the  workstation  software  on  topics  of 
overall  workload,  task  difficulty,  time  pressure,  performance,  mental  and  sensory 
effort,  physical  effort,  frustration  level,and  fatigue.  Figures  9  and  10  show  reprsen- 
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titive  examples  of  the  task  rating  bars.  The  shaded  areas  could  be  changed  by 
“dragging”  with  the  mouse. 

2.1  Statistical  Design:  Each  group  of  three  subjects  were  administered 
three  repeated  trials  and  were  thus  ‘Irained”  with  either  structures  I  or  II  and  with 
either  two  (L)  or  three  (H)  pipes  exiting  each  tank .  The  fourth  trial  was  a  different 


structure  and/or  pipe  count  and  structure  III  was  in  as  one  of  the  options.  The  ex¬ 
perimental  design  plan  is  shown  in  Figure  11. 

Statistical  investigations  were  conducted  using  AXBXCX(DXs)  ANOVA’s 
with  repeated  trials  for  survey  topics,  communication  and  control  frequency.  For  the 
performance  indices  and  the  group  control  frequencies,  a  AXBX(CXs)  design  was 

PIPE  COUNT 

COMMUNICATION/ 

OBSERVABILITY^ 

CONTROLLABILITY 
STRUCTURE 

12  3  4 


TRIALS 

Figure  11.  Design  Overview 

used  with  repeated  trials. 

The  operators’  actions  were  recorded,  and  skill  acquisition  was  assessed 
for  a  total  of  four  trial  units:  three  trials  per  group  (the  acquisition  of  skills  trials) 
across  two  levels  of  pipe/pump  count  and  two  communication  structures,  one  trial 
(the  adaptability  trial)  across  two  levels  of  pipe/pump  count,  two  communication 
structures,  and  two  observability/controllability  structures.  Each  operator’s  mes¬ 
sages  sent  to  other  members  of  the  network  were  recorded  and  the  individual 
aquisition  (subject  stations  A,  B,  and  C)  along  with  group  acquisition  of  situation 
assessment  skills  and  performance  feedback  skills  under  various  forms  of  partition¬ 
ing  and  complexity  were  assessed.  The  process  control  states  and  control  inputs 
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were  recorded  to  assess  individual  and  group  performance  results  for  controlling 
the  process  states  as  directed  by  the  experimenters. 


3.0  SUMMARY  OF  RESULTS 

The  subjects’  responses  to  a  post-trial  survey,  administered  after  each  trial 
showed  that  the  groups  trained  with  low  pipe/pump  counts  perceived  their  work¬ 
loads  higher  than  the  groups  that  trained  with  high  pipe/pump  counts.  Individuals  at 
subject  station  B,  the  “hub,”  for  the  chain  communication  groups  (structure  II) 
showed  much  higher  ratings  than  stations  A  and  C  and  this  difference  was  accentu¬ 
ated  when  taken  to  the  fourth  trial  that  administered  a  redistributed  set  of  observ¬ 
ables  and  controllables  with  the  chain  communication  structure  (structure  III).  This 
survey  showed  that  the  perceived  task  difficulty  means  for  individuals  at  subject 
station  A  decreased  with  repeated  trails,  but,  at  subject  station  B  the  ratings  in¬ 
creased  with  repeated  trials.  For  all  groups,  those  administered  structure  III  gave  a 
lower  mean  task  difficulty  rating  than  those  administered  their  opposite  communica¬ 
tion  structure.  This  result  could  probably  be  accounted  for  by  the  newness  of  the 
observability  and  controllability  structure  and  the  accompanying  skill  transfer  as¬ 
pects.  The  mean  time  pressure  ratings  appear  to  have  bottomed-out  by  the  second 
trial  for  the  groups  that  trained  with  the  circle  communication  structure  (stmcture  I) 
and  the  low  pipe/pump  count  scenarios  and  chain  (structure  II)  with  high  pipe/pump 
count  scenarios,  yet  appear  to  peak  for  the  circle  groups  with  high  pipe/pump 
counts  and  chain  with  low  pipe/pump  counts.  This  result  could  be  accounted  for  by 
the  continued  higher  level  of  task  loading  to  which  the  latter  groups  were  subjected. 

Mental  and  sensory  effort  ratings  grew  with  repeated  trials  for  the  circle 
groups,  but  peaked  at  the  second  trial  for  the  chain  groups.  Individuals  at  subject 
station  B  of  the  chain  groups  logged  less  mean  frustration  than  A  or  C  of  those 
groups:  subject  B  of  the  circle  groups  was  more  frustrated  than  A  or  C  of  those 
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groups.  For  the  fourth  trial,  this  continued  for  chains  given  structure  III,  and  for 
circles  given  either  structure  11  or  111.  Groups  that  trained  with  the  chain  structure 
and  then  administered  the  circle  structure  for  the  fourth  trial  showed  that  individuals 
at  subject  station  B  had  very  high  frustration  levels  compared  to  A  or  C  of  those 
groups.  For  all  groups,  fatigue  grew  with  trials.  The  perceived  performance  was 
higher  for  the  low  pipe/pump  count  groups  than  the  high  pipe/pump  count  groups. 
Individuals  at  subject  station  B  of  the  chain  groups  gave  the  highest  perceived 
performance  ratings  of  the  chain  groups.  As  trials  were  repeated,  the  perceived 
performance  increased  markedly  for  the  chain  groups.  This  trend  of  the  chain 
groups  continued  for  the  fourth  trial  when  a  circle  communication  structure  was 
imposed  on  them.  By  separating  training  groups  by  those  that  had  either  high  or 
low  pipe/pump  count  scenarios,  it  was  seen  that  those  that  had  the  low  pipe/pump 
scenarios  felt  they  had  done  better  than  those  that  had  the  high  pipe/pump  count 
scenarios  when  given  a  scenario  variation  (structure  III  or  the  communication  struc¬ 
ture  opposite)  for  the  fourth  trial. 

The  actual  performance  index  for  each  trial,  for  each  group,  was  recorded 
as  a  function  of  simulation  time.  As  illustrated  in  Figures  1 2  and  1 3  these  functions 
showed  that  for  the  first  three  trials,  both  the  first  derivatives  of  the  functions  are 
decreased  with  time  and  the  overall  function  decreased  with  trials.  For  the  first  three 
trials,  the  mean  of  the  final  time  performance  index  ,  Pl(t,),  was  lower  (better)  for 
the  low  pipe  count  groups  when  compared  with  the  high  pipe  count  groups.  This 
result  is  shown  in  Figures  1 4  and  1 5.  For  the  first  three  trials,  the  chain  groups  had 
lower  mean  Pl(t,)  than  the  circle  groups.  The  chain  groups  showed  a  marked 
decline  in  the  Pl(t,)  with  trials  compared  with  the  circle  groups.  Those  groups  that 
trained  with  low  pipe  counts  started  with  low  mean  values  compared  to  the  those 
trained  with  high  pipe  counts  for  the  first  trial  and  their  Pl(t,)  value  continued  to 
decline  with  trials.  Those  groups  that  trained  with  high  pipe  counts  started  with  high 
mean  values  compared  to  the  those  trained  with  low  pipe  counts  for  the  first  trial 
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Figure  14.  Means  and  One  Standard  Deviation  Bars  for  the  High  Pipe  Count  Groups 
for  each  Trial  (1st,  2nd,  and  3rd)  of  the  Final  Time  Performance  Index 


Figure  15.  Means  and  One  Standard  Deviation  Bars  for  the  Low  Pipe  Count  Groups 
for  each  Trial  (1st,  2nd,  and  3rd)  of  the  Final  Time  Performance  Index 
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and  their  Pl(t,)  means  continued  to  decline  with  trials  but  not  as  low  as  the  low  pipe 
count  groups,  "^or  the  fourth  trial,  those  given  the  opposite  communication  structure 
did  better  (lower  Pl(t,))  than  those  given  structure  III.  Thus,  the  observability/control- 
iability  re-partitioning  had  a  profound  effect.  The  chain  groups  did  better  when 
given  the  chain  communication  structure  for  their  fourth  trial  than  the  circle  groups 
when  given  the  circle  communication  structure. 


The  communication  activity  was  recorded  as  both  a  frequency  count  and  a 
serial  connectivity  (communication  transition  matrix)  between  message  types  for 
each  subject  for  each  trial.  For  the  first  three  trials,  the  communication  transition 


matrices  become  more  dense,  on  average,  with  each  trial.  For  those  trained  with 
circle  groups,  the  S(uggestion)  message  classification  was  the  strongest  way  in 
which  an  individual  could  assert  himself  or  herself  to  the  point  of  evolving  into  a 
leadership  role  (see  Table  1 ).  Those  individuals  at  subject  station  B  who  trained 


with  circle  groups  and  had  many  “suggestions"  in  turn  had  many  “orders”  in  the 
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Table  1.  Group  1:  Communication  Action  Transition  Matrices 
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fourth  trial.  Some  individuals  of  the  circle  groups  at  subject  station  C  that  may  have 
been  evolving  into  a  leadership  role  were  not  allowed  to  have  an  O(rder)  classifica¬ 
tion  when  administered  a  structure  III  scenario  for  a  fourth  trial.  As  illustrated  in 
Tablas  2  and  3  and  in  Figures  16  and  17,  the  circle  groups  communicated  less 
frequently  than  the  chain  groups.  It  is  noted  that  Bavelas  achieved  the  opposite 
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Table  2.  The  Communication  Structure—Pipe/Pump  Count  (AB)  Incidence  Table 
for  the  Message  Frequency 
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Table  3.  The  Communication  Structure— Subject  Station  (AC)  Incidence  Table 
for  the  Message  Frequency 
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Numbar  of  Messages 


Figure  16.  Message  Frequency  Means  and  One  Standard  Deviation  bars  for  ail 
Groups  for  all  Subjects  for  Each  Trial  (1st,  2n,  3rd) 


Figure  17.  Message  Frequency  Means  and  One  Standard  Deviation  bars  for 
the  Fourth  Trial  per  Subject 
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result  in  static  scenario  studies.  The  individuals  at  subject  station  B  for  the  chain 
groups,  on  average,  communicated  twice  as  frequently  as  the  other  two  chain  group 
member  mean  values.  The  mean  message  frequency  for  the  groups  trained  with 
the  circle  communication  structure  was  much  less  than  the  groups  trained  with  the 
chain  structure  when  administered  a  variation  for  the  fourth  trial.  Both  groups  com¬ 
municated  more,  on  average,  when  administered  structure  III  than  when  admini¬ 
stered  their  opposite  communication  structure.  This  could  be  accounted  for  by  the 
newness  of  the  fourth  trial  structure. 


The  control  (pipe/pump  flow  change)  activity  was  recorded  as  both  fre¬ 
quency  count  and  serial  connectivity  between  pipes  changed  for  each  subject  for 
each  trial.  The  serial  connectivity  was  recorded  in  subject  submatrices,  except  for 
structure  III,  where  subject  C  had  a  full  matrix  description.  For  the  first  three  trials, 
the  action  transition  blocks  become  more  dense,  on  average,  with  each  trial.  Only 
the  circle  groups  with  low  pipe  scenarios  showed  blocks  getting  less  dense 
(sparser)  by  the  third  trial  (see  Table  4).  The  pipe  flow  change  frequency  means  of 


all  other  groups  increased  with  trials.  For  the  fourth  trial,  the  groups  that  trained 
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Table  4.  Group  1:  Action  Transition  Matrices 


56 


with  the  circle  communication  structure  (I)  were  administered  a  chain  communica¬ 
tion  structure  (II)  and  their  control  activity,  on  average,  was  at  the  same  mean  as 
the  peak  of  the  second  trial.  For  the  fourth  trial,  the  groups  that  trained  with  the 
chain  communication  structure  (II)  were  administered  a  circle  communication  struc¬ 
ture  (I)  and  their  control  activity,  on  average,  was  below  the  means  of  the  first  three 
trials.  For  all  groups  administered  structure  III,  their  mean  control  activity  was  much 
less  than  in  any  of  their  first  three  trials.  The  reasons  for  this  significant  drop  in¬ 
cluded  both  added  group  loading  for  reorientation  and  a  through-put  constraint 
because  only  subject  C  could  make  pipe/pump  flow  changes.  When  administered 
structure  III  for  the  fourth  trial,  those  groups  that  trained  with  structure  I  showed  less 
activity  than  those  groups  that  trained  with  structure  II.  The  reason  for  this  differ¬ 
ence  appears  to  be  that  structures  II  and  ill  share  obvious  communication  struc¬ 
tures.  A  short  term  control  strategy  was  defined  as  changing  only  pipes  directly 
connected  to  sinks.  A  long  term  strategy  was  defined  as  being  beyond  short  term 
strategies  since  they  included  over  and  underrlow  concerns  in  achieving  a  well 
regulated  steady-state  flow.  From  a  review  of  the  action  transition  block,  it  ap¬ 
peared  that  the  circle  groups  tended  to  evolve  from  short  term  strategies  of  regula¬ 
tion  to  long  term  strategies  by  trial  3,  while  chain  groups  tended  to  stay  with  short 
term  strategies  for  every  trial.  But,  for  some  of  the  reasons  pointed  out  by  Moray,  et 
al,  1986,  the  communication  and  pipe  flow  change  action  transition  matrices 
showed  great  individual  variations  in  behavior. 
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4.0  DISCUSSION  AND  CONCLUSIONS 

The  design  of  this  problem-solving  situation  was  intended  to  provide  a 
“testbed”  approach  such  that  information  gained  from  these  experiments  might  be 
applied  to  such  diverse  areas  as  the  optimal  allocation  of  raw  material  and  man/ 
machine  power  (or  energy)  in  a  hierarchal,  manufacturing  environment  or  to  the 
optimal  assignment  of  troops  and  equipment  in  a  battlefield  environment. 

By  synthesizing  major  research  results  of  others  in  these  areas  and  extend¬ 
ing  the  investigation  to  networks  and  hierarchies,  our  data  have  suggested  that  for 
this  abstract,  yet  tangible,  dynamic  scenario,  chain,  or  linearly  connected,  groups 
obtain  control  skills  faster  and  communicate  more  frequently  when  compared 
against  circle,  or  star,  groups.  An  increase  in  pipe  selection  options  for  control  input 
paths,  intended  to  increase  system  flexibility  for  achieving  group  optimal  transient 
and  steady-state  control,  actually  appeared  to  cause  a  decrease  in  performance. 
We  believe  the  evidence  suggests  this  result  was  related  to  overburdening  the 
subjects’  decision  information  processing  capability. 

The  summary  of  the  major  findings  of  this  research  are;  1 )  chain  groups 
reached  their  respective  “steady-state”  final  time  performance  values  in  fewer  trials; 
2)  circle  groups  had  sparser  communication  transition  matrices;  3)  chain  groups 
tended  to  focus  on  short  term  control  strategies;  4)  too  much  of  an  increase  in  the 
number  of  available  control  input  path  options  will  decrease  group  performance  by 
overloading  the  operators  even  though  this  increase  permitted  more  potential  regu¬ 
lation  solution  sets  ;  and  5)  the  very  communicative  individuals  in  the  “hub”  of  the 
chain  groups  rate  their  group  performances  higher  and  the  “negative”  topics  lower 
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than  the  other  two  group  members. 
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